Papers with toxicity avoidance
Gradient-based Constrained Sampling from Language Models (2022.emnlp-main)
Copied to clipboard
| Challenge: | Large pretrained language models are successful at generating fluent text but are notoriously hard to controllably sample from. |
| Approach: | They propose a sampling procedure that combines the log-likelihood of the language model with arbitrary constraints in a single energy function and then generates samples in . non-autoregressive manner. |
| Outcome: | The proposed method improves on text generation with soft and hard constraints and keyword-guided generation. |